1,848 research outputs found
Supervised Learning in Spiking Neural Networks with Phase-Change Memory Synapses
Spiking neural networks (SNN) are artificial computational models that have
been inspired by the brain's ability to naturally encode and process
information in the time domain. The added temporal dimension is believed to
render them more computationally efficient than the conventional artificial
neural networks, though their full computational capabilities are yet to be
explored. Recently, computational memory architectures based on non-volatile
memory crossbar arrays have shown great promise to implement parallel
computations in artificial and spiking neural networks. In this work, we
experimentally demonstrate for the first time, the feasibility to realize
high-performance event-driven in-situ supervised learning systems using
nanoscale and stochastic phase-change synapses. Our SNN is trained to recognize
audio signals of alphabets encoded using spikes in the time domain and to
generate spike trains at precise time instances to represent the pixel
intensities of their corresponding images. Moreover, with a statistical model
capturing the experimental behavior of the devices, we investigate
architectural and systems-level solutions for improving the training and
inference performance of our computational memory-based system. Combining the
computational potential of supervised SNNs with the parallel compute power of
computational memory, the work paves the way for next-generation of efficient
brain-inspired systems
Monatomic phase change memory
Phase change memory has been developed into a mature technology capable of
storing information in a fast and non-volatile way, with potential for
neuromorphic computing applications. However, its future impact in electronics
depends crucially on how the materials at the core of this technology adapt to
the requirements arising from continued scaling towards higher device
densities. A common strategy to finetune the properties of phase change memory
materials, reaching reasonable thermal stability in optical data storage,
relies on mixing precise amounts of different dopants, resulting often in
quaternary or even more complicated compounds. Here we show how the simplest
material imaginable, a single element (in this case, antimony), can become a
valid alternative when confined in extremely small volumes. This compositional
simplification eliminates problems related to unwanted deviations from the
optimized stoichiometry in the switching volume, which become increasingly
pressing when devices are aggressively miniaturized. Removing compositional
optimization issues may allow one to capitalize on nanosize effects in
information storage
ESSOP: Efficient and Scalable Stochastic Outer Product Architecture for Deep Learning
Deep neural networks (DNNs) have surpassed human-level accuracy in a variety
of cognitive tasks but at the cost of significant memory/time requirements in
DNN training. This limits their deployment in energy and memory limited
applications that require real-time learning. Matrix-vector multiplications
(MVM) and vector-vector outer product (VVOP) are the two most expensive
operations associated with the training of DNNs. Strategies to improve the
efficiency of MVM computation in hardware have been demonstrated with minimal
impact on training accuracy. However, the VVOP computation remains a relatively
less explored bottleneck even with the aforementioned strategies. Stochastic
computing (SC) has been proposed to improve the efficiency of VVOP computation
but on relatively shallow networks with bounded activation functions and
floating-point (FP) scaling of activation gradients. In this paper, we propose
ESSOP, an efficient and scalable stochastic outer product architecture based on
the SC paradigm. We introduce efficient techniques to generalize SC for weight
update computation in DNNs with the unbounded activation functions (e.g.,
ReLU), required by many state-of-the-art networks. Our architecture reduces the
computational cost by re-using random numbers and replacing certain FP
multiplication operations by bit shift scaling. We show that the ResNet-32
network with 33 convolution layers and a fully-connected layer can be trained
with ESSOP on the CIFAR-10 dataset to achieve baseline comparable accuracy.
Hardware design of ESSOP at 14nm technology node shows that, compared to a
highly pipelined FP16 multiplier design, ESSOP is 82.2% and 93.7% better in
energy and area efficiency respectively for outer product computation.Comment: 5 pages. 5 figures. Accepted at ISCAS 2020 for publicatio
A flexible and fast PyTorch toolkit for simulating training and inference on analog crossbar arrays
We introduce the IBM Analog Hardware Acceleration Kit, a new and first of a
kind open source toolkit to simulate analog crossbar arrays in a convenient
fashion from within PyTorch (freely available at
https://github.com/IBM/aihwkit). The toolkit is under active development and is
centered around the concept of an "analog tile" which captures the computations
performed on a crossbar array. Analog tiles are building blocks that can be
used to extend existing network modules with analog components and compose
arbitrary artificial neural networks (ANNs) using the flexibility of the
PyTorch framework. Analog tiles can be conveniently configured to emulate a
plethora of different analog hardware characteristics and their non-idealities,
such as device-to-device and cycle-to-cycle variations, resistive device
response curves, and weight and output noise. Additionally, the toolkit makes
it possible to design custom unit cell configurations and to use advanced
analog optimization algorithms such as Tiki-Taka. Moreover, the backward and
update behavior can be set to "ideal" to enable hardware-aware training
features for chips that target inference acceleration only. To evaluate the
inference accuracy of such chips over time, we provide statistical programming
noise and drift models calibrated on phase-change memory hardware. Our new
toolkit is fully GPU accelerated and can be used to conveniently estimate the
impact of material properties and non-idealities of future analog technology on
the accuracy for arbitrary ANNs.Comment: Submitted to AICAS202
- …